Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Faster/Better ens. B processing/recentering #594

Merged
merged 10 commits into from
Sep 8, 2023
Merged

Faster/Better ens. B processing/recentering #594

merged 10 commits into from
Sep 8, 2023

Conversation

guillaumevernieres
Copy link
Contributor

Description

Features added:

  • replaced multiple steps in the workflow by 1 executable, limiting the io to a bare minimum and decreasing the job wall-time from close to 30mn donw to ~10mn. Still too slow, but better!
  • addition of background error related diagnostics (explained variance by the physical balance, unbalanced bkg-err, ...)
  • the new field are plotted in the verify task

waiting for oops/pull/2340

DON'T trigger the CI for this draft PR just yet, it's pointing to jcsda-internal for oops.

CoryMartin-NOAA
CoryMartin-NOAA reviewed Aug 30, 2023
View reviewed changes
Copy link
Contributor

@CoryMartin-NOAA CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor comments/questions

scripts/exgdas_global_marine_analysis_post.py Show resolved Hide resolved
scripts/exgdas_global_marine_analysis_post.py Show resolved Hide resolved
utils/soca/gdas_ens_handler.h Outdated Show resolved Hide resolved
@guillaumevernieres
Copy link
Contributor Author

@ShastriPaturi and @AndrewEichmann-NOAA , when you have time, please review.

@guillaumevernieres guillaumevernieres marked this pull request as ready for review September 8, 2023 15:17
@guillaumevernieres guillaumevernieres added hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion labels Sep 8, 2023
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed hera-GW-RT Queue for automated testing with global-workflow on Hera orion-GW-RT Queue for automated testing with global-workflow on Orion labels Sep 8, 2023
AndrewEichmann-NOAA
AndrewEichmann-NOAA previously approved these changes Sep 8, 2023
View reviewed changes
scripts/exgdas_global_marine_analysis_post.py Show resolved Hide resolved
scripts/exgdas_global_marine_analysis_vrfy.py Outdated Show resolved Hide resolved
ShastriPaturi
ShastriPaturi previously approved these changes Sep 8, 2023
View reviewed changes
Copy link
Collaborator

@ShastriPaturi ShastriPaturi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@guillaumevernieres guillaumevernieres mentioned this pull request Sep 8, 2023
Closed
@emcbot
Copy link

emcbot commented Sep 8, 2023

Automated Global-Workflow GDASApp Testing Results:
Machine: hera

Start: Fri Sep  8 17:04:17 UTC 2023 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Sep  8 18:08:53 UTC 2023
---------------------------------------------------
Tests:                                 *SUCCESS*
Tests: Completed at Fri Sep  8 18:52:09 UTC 2023
Tests: 100% tests passed, 0 tests failed out of 50

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Sep 8, 2023
@emcbot
Copy link

emcbot commented Sep 8, 2023

Automated Global-Workflow GDASApp Testing Results:
Machine: orion

Start: Fri Sep  8 12:02:52 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Fri Sep  8 13:27:47 CDT 2023
---------------------------------------------------
Tests:                                 *SUCCESS*
Tests: Completed at Fri Sep  8 14:03:03 CDT 2023
Tests: 100% tests passed, 0 tests failed out of 50

@emcbot emcbot added orion-GW-RT-Passed Automated testing with global-workflow successful on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Sep 8, 2023
@guillaumevernieres
Copy link
Contributor Author

I can't remember the last time the ci was successful the first time around! I'll take it :)
Can I have a re-review @CoryMartin-NOAA , @ShastriPaturi and @AndrewEichmann-NOAA ?

AndrewEichmann-NOAA
AndrewEichmann-NOAA approved these changes Sep 8, 2023
View reviewed changes
Copy link
Collaborator

@AndrewEichmann-NOAA AndrewEichmann-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks reasonable

@guillaumevernieres guillaumevernieres merged commit 9c552c0 into develop Sep 8, 2023
9 checks passed
@guillaumevernieres guillaumevernieres deleted the feature/faster-better-recentering branch September 11, 2023 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AndrewEichmann-NOAA AndrewEichmann-NOAA AndrewEichmann-NOAA approved these changes

@CoryMartin-NOAA CoryMartin-NOAA Awaiting requested review from CoryMartin-NOAA

@ShastriPaturi ShastriPaturi Awaiting requested review from ShastriPaturi

Assignees
No one assigned
Labels
hera-GW-RT-Passed Automated testing with global-workflow successful on Hera orion-GW-RT-Passed Automated testing with global-workflow successful on Orion
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@guillaumevernieres @emcbot @CoryMartin-NOAA @ShastriPaturi @AndrewEichmann-NOAA